Reinforcement Learning for Mobile Robot Collision Avoidance in Navigation Tasks

Collision avoidance is fundamental for mobile robot navigation. In general, its solutions include: {\it map-based} and {\it mapless approaches.} In the map-based approach, robots pre-plan collision-free paths based on an environment map and follow their paths during navigation. On the other hand, the mapless approach requires robots to avoid collisions without referencing to an environment map. This thesis first studies the map-based approach for multiple robots to collectively build environment maps. In this study, a robot following a pre-planned path may encounter unexpected obstacles, such as other moving robots and obstacles inaccurately presented on an environment map. This motivates us to study mapless collision avoidance in the second part of the thesis. Mapless collision avoidance requires a robot to infer an optimal action based on sensor data and operate in real time. Inferring an optimal action in a timely manner is computationally expensive, particularly when a robot has limited on-board computing resources. To avoid the expensive online action inferring, this thesis presents a reinforcement learning approach which learns policies for mapless collision avoidance under real-world settings. We first propose a Real-Time Actor-Critic Architecture (RTAC) to support asynchronous reinforcement learning under real-time constraint. Based on RTAC, we propose asynchronous reinforcement learning methods for mapless collision avoidance of various numbers of robots under different environment configurations. Through extensive experiments, we demonstrate that RTAC serves as a solid foundation to support multi-task and multi-agent learning for mapless collision avoidance under asynchronous settings

CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms

How to optimally dispatch orders to vehicles and how to tradeoff between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a large-scale parallel ranking problem and study the joint decision-making task of order dispatching and fleet management in online ride-hailing platforms. This task brings unique challenges in the following four aspects. First, to facilitate a huge number of vehicles to act and learn efficiently and robustly, we treat each region cell as an agent and build a multi-agent reinforcement learning framework. Second, to coordinate the agents from different regions to achieve long-term benefits, we leverage the geographical hierarchy of the region grids to perform hierarchical reinforcement learning. Third, to deal with the heterogeneous and variant action space for joint order dispatching and fleet management, we design the action as the ranking weight vector to rank and select the specific order or the fleet management destination in a unified formulation. Fourth, to achieve the multi-scale ride-hailing platform, we conduct the decision-making process in a hierarchical way where a multi-head attention mechanism is utilized to incorporate the impacts of neighbor agents and capture the key agent in each scale. The whole novel framework is named as CoRide. Extensive experiments based on multiple cities real-world data as well as analytic synthetic data demonstrate that CoRide provides superior performance in terms of platform revenue and user experience in the task of city-wide hybrid order dispatching and fleet management over strong baselines.Comment: CIKM 201

Programming Bacteria With Light—Sensors and Applications in Synthetic Biology

Photo-receptors are widely present in both prokaryotic and eukaryotic cells, which serves as the foundation of tuning cell behaviors with light. While practices in eukaryotic cells have been relatively established, trials in bacterial cells have only been emerging in the past few years. A number of light sensors have been engineered in bacteria cells and most of them fall into the categories of two-component and one-component systems. Such a sensor toolbox has enabled practices in controlling synthetic circuits at the level of transcription and protein activity which is a major topic in synthetic biology, according to the central dogma. Additionally, engineered light sensors and practices of tuning synthetic circuits have served as a foundation for achieving light based real-time feedback control. Here, we review programming bacteria cells with light, introducing engineered light sensors in bacteria and their applications, including tuning synthetic circuits and achieving feedback controls over microbial cell culture

Abnormal hubs in global network as potential neuroimaging marker in generalized anxiety disorder at rest

BackgroundMounting studies have reported altered neuroimaging features in generalized anxiety disorder (GAD). However, little is known about changes in degree centrality (DC) as an effective diagnostic method for GAD. Therefore, we aimed to explore the abnormality of DCs and whether these features can be used in the diagnosis of GAD.MethodsForty-one GAD patients and 45 healthy controls participated in the study. Imaging data were analyzed using DC and receiver operating characteristic (ROC) methods.ResultsCompared with the control group, increased DC values in bilateral cerebellum and left middle temporal gyrus (MTG), and decreased DC values in the left medial frontal orbital gyrus (MFOG), fusiform gyrus (FG), and bilateral posterior cingulate cortex (PCC). The ROC results showed that the DC value of the left MTG could serve as a potential neuroimaging marker with high sensitivity and specificity for distinguishing patients from healthy controls.ConclusionOur findings demonstrate that abnormal DCs in the left MTG can be observed in GAD, highlighting the importance of GAD pathophysiology

Stimulated thyroglobulin and pre-ablation antithyroglobulin antibody products can predict the response to radioiodine therapy of TgAb-positive differentiated thyroid cancer patients: a retrospective study

ObjectiveWe aimed to explore the predictive value of stimulated thyroglobulin (sTg) and pre-ablation antithyroglobulin (pa-TgAb) products for the effect of radioiodine therapy (RAIT) on TgAb-positive differentiated thyroid cancer (DTC) patients.MethodsIn this study, we enrolled 265 patients with TgAb-positive DTC who underwent RAIT after total thyroidectomy (TT). Based on the last follow-up result, the patients were divided into two groups: the excellent response (ER) group and the non-excellent response (NER) group. We analyzed the factors related to the effect of RAIT.ResultsThe ER group consisted of 197 patients. The NER group consisted of 68 patients. For the univariate analysis, we found that the maximal tumor diameter, whether with extrathyroidal extension (ETE), bilateral or unilateral primary lesion, multifocality, preoperative TgAb (preop-TgAb), pa-TgAb, sTg × pa-TgAb, initial RAIT dose, N stage, and surgical extent (modified radical neck dissection or not), showed significant differences between the ER group and NER group (all p-values <0.05). The receiver operating characteristic (ROC) curves showed that the cutoff value was 724.25 IU/ml, 424.00 IU/ml, and 59.73 for preop-TgAb, pa-TgAb, and sTg × pa-TgAb, respectively. The multivariate logistic regression analysis results indicated that pa-TgAb, sTg × pa-TgAb, initial RAIT dose, and N stage were independent risk factors for NER (all p-values <0.05). For the Kaplan–Meier analysis of disease-free survival (DFS), the median DFS of the patients with sTg × pa-TgAb < 59.73 and initial RAIT dose ≤ 100 mCi was significantly longer than that of the patients with sTg × pa-TgAb ≥ 59.73 (50.27 months vs. 48.59 months, p = 0.041) and initial RAIT dose >100 mCi (50.50 months vs. 38.00 months, p = 0.030).ConclusionWe found the sTg and pa-TgAb conducts is a good predictor of the efficacy of RAIT in TgAb-positive DTC patients. It can play a very positive and important role in optimizing treatment, improving prognosis, and reducing the burden of patients

Synergistic effects of atomic oxygen and thermal cycling in low earth orbit on polymer-matrixed space material

Polymer-matrixed materials are widely used in the spacecrafts' structures. However, crafts located in the LEO（Low Earth Orbit） would suffer from hazardous environment factors when orbiting in the space. It has been reported that the space environment factors’ integral effect (which represents the factual detriment in space) is not equivalent to the simple summation of each individual. Hence, atomic oxygen and thermal cycling were selected as the starting point for studying the typical LEO synergistic effects on polymer-matrixed space material. In this work, methods such as surface morphology observation, surface components analyzation and inter-laminar-shear strength test were embraced to gather the basic information for the study of degradation. As a result, focusing on the composites selected in this work, synergistic effects do exist between the two factors (AO&TC, representing for atomic oxygen and thermal cycling combined). Besides, a quantified index was proposed to represent synergistic characteristics，so as to lay the foundation for the scientific evolution of material characterization

EBSD Characterization of 7075 Aluminum Alloy and Its Corrosion Behaviors in SRB Marine Environment

Aluminum alloy 7075 is an important engineering material for ship structures. However, the corrosion of Al alloys generally exists in various environments, especially in the marine environment. Currently, the corrosion behaviors of Al alloy 7075 in sulfate-reducing bacteria (SRB) marine environment has not been well-addressed. In this paper, the corrosion effect of SRB on 7075 aluminum alloys was studied by adding SRB to real seawater. The microstructure and grain orientation of the super-hardness Al alloy 7075 were studied via the electron back-scattered diffraction (EBSD)technology, and the electrochemical impedance spectroscopy (EIS) test of the electrochemical corrosion behavior of 7075 in a variety of microorganisms, mainly SRB, in real seawater was continuously performed for 21 days. It was concluded that Al alloy 7075 has the strongest texture intensity on the (001), (111), (010), and (0–10) planes, which is 2.565. Adding SRB to real seawater accelerated the corrosion rate, and after corrosion on the 14th day, the protective film on the 7075 aluminum alloy surface was completely broken, and the impedance was significantly reduced

Decellularized Pig Kidney with a Micro-Nano Secondary Structure Contributes to Tumor Progression in 3D Tumor Model

In spite of many anti-cancer drugs utilized in clinical treatment, cancer is still one of the diseases with the highest morbidity and mortality worldwide, owing to the complexity and heterogeneity of the tumor microenvironment. Compared with conventional 2D tumor models, 3D scaffolds could provide structures and a microenvironment which stimulate native tumor tissues more accurately. The extracellular matrix (ECM) is the main component of the cell in the microenvironment that is mainly composed of three-dimensional nanofibers, which can form nanoscale fiber networks, while the decellularized extracellular matrix (dECM) has been widely applied to engineered scaffolds. In this study, pig kidney was used as the source material to prepare dECM scaffolds. A chemical crosslinking method was used to improve the mechanical properties and other physical characteristics of the decellularized pig kidney-derived scaffold. Furthermore, a human breast cancer cell line (MCF-7) was used to further investigate the biocompatibility of the scaffold to fabricate a tumor model. The results showed that the existence of nanostructures in the scaffold plays an important role in cell adhesion, proliferation, and differentiation. Therefore, the pig kidney-derived matrix scaffold prepared by decellularization could provide more cell attachment sites, which is conducive to cell adhesion and proliferation, physiological activities, and tumor model construction